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Executive Summary 


WestEd is undertaking a series of implementation studies intended 
to inform the California State University (CSU) system about the 
implementation of Executive Order 1110 (EO 1110). A major policy 
adopted by the CSU Chancellor’s Office in 2017, EO 1110 requires 
CSU campuses to eliminate noncredit developmental courses — 
often known as “remedial” courses — in Written Communication 
(WC) and Mathematics/Quantitative Reasoning (QR), change the 
process for how students are placed into WC and QR courses, and 
improve how students are supported to succeed. The policy also 
calls for changes to the system’s Early Start programs that would 
enable students designated as needing additional support in WC or 
QR (designated as placement Categories III and IV) to obtain 
baccalaureate-level credit during the summer prior to their initial 
fall enrollment. This report examines initial outcomes for students 
who enrolled in summer 2019 Early Start mathematics courses that 
were offered to students in response to the policy. On several 
campuses, but not all, the Early Start courses were changed 
significantly from prior years to provide additional supports for 
students as they transitioned to their first year of college. 


Since 2011, the CSU system has designated summer Early Start programs, which generally 
offered some sort of pre-baccalaureate coursework for incoming students, as a key strategy for 
supporting students designated as “not college ready.” EO 1110 asked campuses to revamp 
their Early Start offerings starting in 2019 to provide baccalaureate-level courses in WC and QR 
during the summer. Per the language of the Executive Order, Early Start is “required” for 
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students placed in Category IV (those determined to have the highest need for additional 
academic support in WC or QR) and “recommended” for those in Category III (those 
determined to need some additional academic support). Because not all campuses offered 
courses that met the requirements of the new policy in the summer of 2019, WestEd 
researchers, in close consultation with staff from the CSU Chancellor’s Office, limited this 
report’s analysis to a subset of seven campuses identified as having a robust implementation of 
Early Start in the summer of 2019. In addition, because very few Early Start courses either met 
the General Education (GE) requirement for WC or served as the first course in a two-semester 
WC sequence, the study is limited to focusing on QR courses. This study considers Early Start 
outcomes on seven of the 22 CSU campuses that offer Early Start and, as a result, does not 
draw conclusions about “systemwide” Early Start policy or implementation. 


Although EO 1110 recommends and/or requires that students designated as needing additional 
support in QR attend Early Start in the summer prior to their first fall enrollment, not all 
students do so. As a result, outcomes can be compared for two groups of students: those who 
participated in Early Start and those who were in the same placement category and at the same 
campus but chose not to participate in Early Start. 


This analysis of short-term outcomes of Early Start participation on seven campuses shows that, 
after a single semester, enrolling in the new Early Start courses in QR had a positive impact on 
students’ ability not only to complete a baccalaureate-level math course of three or more units 
but also to complete the GE requirement in QR (referred to as the B4 requirement) by the end 
of the fall term. Additionally, there was a small, statistically significant short-term impact on 
retention for students in Category IV. 


e Students who participated in Early Start across the seven campuses not only attempted 
a math course of three or more units at higher rates than their counterparts who opted 
out of Early Start participation, but they also had higher success rates in those courses. 


- For students who participated in Early Start, 91 percent of students in Category III 
and 83 percent of students in Category IV completed a math course of three or 
more units by the end of fall 2019. For those who did not participate in Early Start, 
53 percent of students in Categories Ill and lV completed a math course of three 
or more units by the end of fall 2019. 


e Students who participated in Early Start at any of the seven campuses included in this 
study completed the B4 requirement by the end of fall 2019 at higher rates than their 
non-Early Start peers in the same QR placement. 


- Seventy-four percent of students in Category Ill who participated in Early Start 
completed the B4 requirement by the end of the fall semester, compared to 29 
percent of Category Ill students who did not participate in Early Start. Similarly, 60 
percent of Category IV students who enrolled in Early Start completed the B4 
requirement by the end of fall 2019, compared to 25 percent of Category IV 
students who did not enroll in Early Start. 
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e In terms of retention to the spring 2020 semester, although there was little difference 
between Category III students who participated in Early Start and those who did not, 
there were statistically significant differences in retention rates for students in 
Category IV who participated in Early Start. 


- Ninety-one percent of students in Category IV who participated in Early Start on 
these seven campuses reenrolled in the spring 2020 term, compared to 87 percent 
of those in the same placement category who did not participate in Early Start. 


Although these short-term outcomes are positive, it is too soon to tell whether these benefits 
will have an impact on retention and completion in the long run. Additionally, this analysis 
cannot determine whether the success in the summer courses was due to the focused time on 
one course, the additional supports provided in the summer, or a difference in the course itself. 
What does seem clear, however, is that allowing students to take baccalaureate-level courses in 
the summer does provide students with an advantage that allows them to make initial short- 
term gains toward fulfilling their GE math requirements. 
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Preliminary Outcomes for 
California State University 
Students in Early Start 
Mathematics 


WestEd is undertaking a multiyear series of implementation studies 
intended to inform the California State University (CSU) system 
about the implementation of Executive Order 1110 (EO 1110). 

A major policy adopted by the CSU Chancellor’s Office in 2017, 

EO 1110 requires CSU campuses to eliminate noncredit 
developmental courses (often known as “remedial” courses) in 
Written Communication (WC) and Mathematics/Quantitative 
Reasoning (QR), change the process for how students are placed 
into WC and QR courses, and improve how students are supported 
to succeed. WestEd’s first EO 1110 report (Bracco et al., 2019) 
describes the variation in course models and instructional 
approaches adopted by campuses in response to this major policy 
change. A second report (Bracco et al., 2020) examines the progress 
of nearly 60,000 students during the policy’s first year of 
implementation, the 2018/19 academic year. The third report 
(Bracco et al., 2021) examines outcomes for students who entered 
the CSU system in 2018 in comparison with those who entered in 
2017, prior to the new policy’s implementation. This fourth report 
examines the initial impacts of participation in redesigned Early 
Start QR courses, a major component of the EO 1110 policy. 
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As discussed in the previous reports in this series, the policy changes enacted by EO 1110 are 
part of a broader effort to improve completion rates in the CSU system, the largest public four- 
year university system in the country, with particular focus on reducing the gap in achievement 
for students from traditionally underrepresented populations. The policy changes (eliminating 
developmental education courses and relying on multiple measures for placement rather than 
single high-stakes tests) draw upon research concerning the impact of developmental 
coursework. Researchers have found that many students who begin their college careers in 
noncredit developmental courses often drop out before enrolling in General Education (GE) 
courses or completing their degrees, and many students who are placed into these courses 
could be successful without them (Burdman, 2017; Rutschow & Mayer, 2018). 


Whereas WestEd’s earlier reports in this series on EO 1110 are focused on outcomes in the 
traditional fall and spring terms, this report looks at another significant change dictated by the 
policy, the summer Early Start programs. EO 1110 called upon campuses to make changes to 
their summer offerings for those students designated as needing support in WC and/or QR. The 
changes to Early Start are intended to help these students earn baccalaureate credit and 
potentially even meet their GE requirement in WC and/or QR in the summer term prior to their 
first fall enrollment term. 


Background on Early Start 


In California and nationally, to help students entering higher education who are not quite ready 
for college-level work, summer programs have long been part of a strategy to improve these 
students’ chances for college success (Barnett et al., 2012; Kurlaender et al., 2018). Although 
the summer programs vary, their shared goal is to provide additional supports, both academic 
and nonacademic, prior to students’ enrollment in a full-semester course load. Researchers 
have found that these programs have had positive short-term outcomes, but long-term impacts 
have not been as evident (Barnett et al., 2012; Scrivener et al., 2018; Kurlaender et al., 2018). 


Since 2011, the CSU system has designated summer Early Start programs as a key strategy for 
supporting students designated as “not college ready.” Although intended to be implemented 
systemwide, Early Start’s exact program components, duration, and intensity have varied 
significantly by campus. Prior to EO 1110, Early Start offerings included a wide range of 
strategies such as intensive summer bridge programs designed to better prepare students for 
the academic and affective demands of college as well as single-unit, one-week online courses 
designed to help students who wished to retake a placement test in order to change their initial 
placement into developmental courses. 


With the implementation of the EO 1110 policy, campuses were asked to revamp their Early 
Start offerings beginning in 2019 to provide baccalaureate-level courses in WC and QR during 
the summer. Per the language of the Executive Order, Early Start is “required” for students 
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placed in Category IV (i.e., those determined to have the highest need for additional academic 
support) and “recommended?” for those in Category III (those determined to need some 
additional academic support).1 To better support the students who enroll in Early Start, the 
policy calls upon campuses to offer one or more of the following options in the summer: 


e Baccalaureate-credit courses that meet the CSU’s GE requirement in WC (referred to as 
the A2 requirement) or in QR (referred to as the B4 requirement) 


e Baccalaureate-level courses that provide the first course of a two-semester (often 
referred to as “stretch”) sequence leading to the A2 or B4 requirement in the next 
semester 


e Baccalaureate courses that prepare students for fall courses that satisfy the A2 or B4 
requirement (the summer courses could be baccalaureate-level, non-GE courses and be 
offered with up to two units of concurrent pre-baccalaureate support in QR or WC) 


“Effective summer 2019, Early Start Programs shall offer 
primarily baccalaureate credit-bearing general education written 
communication and mathematics/quantitative reasoning 
courses, and those courses shall be offered in sufficient numbers 
to meet student demand. Instructional content considered pre- 
baccalaureate will carry a maximum of two units and shall be 
offered concurrently with a college-level, baccalaureate credit- 


bearing course.”” 


Campus-level concerns prior to Early Start modifications 


In the fall of 2018, WestEd researchers met with faculty and administrators on nine CSU 
campuses to understand how the implementation of EO 1110 was proceeding (Bracco et al., 
2019). At that time, most campuses had not determined exactly how they were going to 
revamp their Early Start offerings to meet the requirements of the new policy. Many campus 
representatives did not think the prior iteration of Early Start had been very impactful, 
particularly the one-week, one-unit courses. Some campuses had more robust program 


1 See Box One, page 11, for additional detail on the placement categories. 
2 CSU Executive Order 1110 Frequently Asked Questions (FAQ). Updated: May 10, 2019. https://www2.calstate.edu/csu- 
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offerings that were focused on a smaller number of students, including bridge programs that 
aimed not only to shore up academic skills but also to familiarize students with campus policies 
and procedures and acclimate them to campus life. Campus teams noted the importance of 
supporting students in a transition to campus life, providing information on how to access 
campus-based resources, build peer communities, and manage multiple new time 
commitments. While changes to Early Start for summer 2019 were still in early planning stages 
at the time of these visits, campus interviewees expressed some concerns about the Early Start 
requirement and its potential to be helpful. 


e Campus representatives expressed concerns that the Early Start requirement could 
send mixed messages to students about their college readiness or create financial or 
other burdens by requiring students to attend college in the summer. In particular, 
interviewees noted that many students work during the summer, and the Early Start 
requirement could inhibit their ability to save money needed for the academic year. 


“Our students, generally speaking, are first-generation college 
students. Summer courses are way more expensive for them 
than other courses, and summer is when a lot of our students 
tend to make their money. And so, for them to try to come to 
campus from wherever, it’s expensive for them, and not just 
transportation-wise, but because they’re missing hours of 
work.” 


— CSU campus faculty member 2018 


e Both WC and QR instructors questioned whether GE course content was best offered in 
a condensed summer format. Faculty particularly questioned whether the shorter 
summer time frame provided adequate time for STEM students to gain the skills 
necessary to succeed in future coursework. The two-semester stretch model (whereby 
the traditional one semester course is “stretched” over two semesters) can also be 
problematic to begin in the summer, particularly if a student takes an Early Start stretch 
course at one campus and then enrolls in the fall at a different campus where the 
courses are not aligned with those of the first campus. 


Interviewees on many campuses wanted the flexibility to design holistic programming 
that would help prepare students to succeed at their campuses, including addressing 
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affective dimensions such as study skills, critical self-reflection around disciplinary 
content, and orientation to specific support resources at destination campuses. 


“Summer Bridge programs have demonstrated an opportunity to 
really transition students, particularly first-generation, lowest- 
performing, to come to the university, to get oriented to 
campus, to build a community, to get associated with mentors 
and other faculty. Unfortunately, because the Early Start 
program is so massive, it’s just really impossible to do that.” 


— CSU campus faculty member 2018 


Interviewees also noted the different contexts of the various CSU campuses and wanted more 
flexibility to determine Early Start programming that works for their individual students. 
Subsequent to researchers’ fall 2018 visits, the Chancellor’s Office did allow for additional 
flexibility for Early Start in summer 2019. Although students designated as Category IV were still 
technically required to enroll in Early Start, there would be no penalty for not doing so. And 
although campuses were encouraged to offer baccalaureate-level courses and courses that met 
A2 or B4 requirements (or contributed to meeting those requirements), campuses were 
allowed to offer a baccalaureate-level elective course (one or more units) concurrently with 
pre-baccalaureate support units (two units maximum) for either WC or QR. 


Methodology 


The variation in Early Start programming across campuses has been considerable since the 
inception of the program. Because not all campuses offered courses that met the requirements 
of the new policy in the summer of 2019, WestEd researchers, in close consultation with staff 
from the CSU Chancellor’s Office, limited this analysis to a subset of campuses. This report 
focuses on initial outcomes for students who participated in Early Start on seven campuses that 
were identified as having a robust implementation of Early Start in summer 2019. Therefore, 
this analysis examines programming most aligned with the policy at that time, and the study’s 
results cannot be generalized to the other campuses; also, Early Start offerings have continued 
to evolve since summer 2019. In addition, because very few Early Start courses in 2019 either 
met the GE requirement for WC or served as the first course in a WC stretch sequence, the 
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study focuses on QR. The seven campuses included in this study? all had at least 100 students 
enrolled in Early Start QR, and all offered baccalaureate-level courses. 


The findings in this report are based on WestEd researchers’ analysis of data for a cohort of 
24,616 students who were entering the CSU system as first-year students in the 2019 fall term 
at seven campuses. The study particularly focuses on the students from this entering cohort 
who were placed in QR Categories Ill and IV at the beginning of the summer, a total of 7,367 
students, and even more specifically, the 7,000 with adequate data for the study. All of these 
students were recommended or required to attend Early Start. The first part of this report 
provides descriptive statistics on their participation rates in Early Start, disaggregated by 
gender, race/ethnicity, declared major, and campus. 


To begin to understand the potential impact of participation in Early Start on these campuses 
under the policy, this study then examines outcomes on several indicators for two groups: 
students who participated in Early Start and a comparison group of students who were 
enrolled at the same campus and also required or recommended to attend Early Start but 
who chose not to participate in Early Start. 


Accordingly, the population of 7,367 students placed in QR Categories IIl and IV at the 
beginning of summer 2019 was narrowed to the 7,000 students who enrolled for the fall 2019 
term and had the full set of demographics and placement information needed to construct the 
two comparable groups: students who did participate in Early Start in summer 2019 and those 
who did not.* Within this sample, 2,559 had enrolled in Early Start in summer 2019 and then 
enrolled in fall 2019 at the same campus where they had been in Early Start. 


Because Early Start is intended to help students fully complete their GE QR requirements by the 
end of their first year, the analysis would ideally focus on outcomes at the end of an entire 
academic year, but the disruptions to the spring 2020 term by COVID-19 introduced a number 
of confounding factors that severely limited researchers’ ability to make analytical inferences 
from student outcomes during the spring term. The analysis therefore focuses on the early 
effects of Early Start participation, specifically 


e completion of a math course of three or more units by the end of the fall 2019 term, 


e completion of the GE requirement in math (also known as the B4 requirement) by the 
end of the fall 2019 term, and 


e reenrollment in the spring 2020 term. 


Descriptive statistics were used to examine the association of Early Start participation with 
math completion in the fall. This study first presents, by QR placement category, the 


3 Campuses included in this analysis are Dominguez Hills, Fullerton, Los Angeles, Northridge, Pomona, San Jose, and Stanislaus. 
Outcomes studied are aggregated rather than broken down by individual campus. 


4 Of the students placed in QR Categories III and IV at the beginning of summer 2019, 367 either enrolled in Early Start at a 
different campus than that at which they enrolled in fall 2019 or had missing demographic or high school information. 
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percentage of students who completed a QR course? with a grade of C- or better in summer 
2019 (for Early Start participants) and in fall 2019 (for Early Start participants compared to non— 
Early Start participants). A chi-square test of independence was used to assess the statistical 
significance of the differences in math completion at the end of the fall term between students 
who enrolled in Early Start and their peers who opted out of Early Start participation. 


Next, the study reports the percentage of students who completed the B4 requirement in 
summer 2019 (for Early Start participants) and in fall 2019 (for Early Start participants 
compared to non-Early Start participants). Again, a chi-square test of independence was used 
to assess the statistical significance of differences in B4 completion between the two groups. 


Finally, the spring 2020 reenrollment rates are reported, by QR placement category, for Early 
Start participants compared to their non—Early Start peers. Again, the analysis applied a chi- 
square test to determine whether any observed difference is statistically significant. 


To account for differences between Early Start participants and non—Early Start participants on 
spring 2020 reenrollment rates, a matching analysis using the Mahalanobis distance metric was 
also performed; students in Categories III and lV who took Early Start math classes in the 
summer were matched to similar students who did not take Early Start classes. Matching 
eliminates the systematic differences in baseline characteristics (such as demographic 
information or placement category) between students who attended Early Start and similar 
students who did not attend, allowing an assessment of the difference between the two groups 
with more precision. Early Start students were matched to non—Early Start students using the 
Mahalanobis distance metric based on the following characteristics: campus of enrollment, QR 
placement category, high school grade point average (GPA), high school math GPA, major of 
admission, age, gender, and ethnicity. Each Early Start student was matched to one most 
similar non—Early Start student, and matching was conducted with replacement (i.e., students 
who did not participate in Early Start were available for consideration as a potential match for 
more than one Early Start student). The final analytic sample consisted of 2,559 Early Start 
students who were matched with 2,559 weighted non—Early Start students.® To evaluate the 
impact of Early Start participation, all matched students were included in a regression model 
that included as covariates the same variables that were used in the matching process. 
Appendix A provides additional details on the matching methodology. 


> Only math courses of three or more units were considered in this analysis. 


° For analysis, each Early Start student received a weight of 1, and each matched non-Early Start student received a weight that 
was proportional to the number of times the student was selected as a match. The total sample sizes of these groups (Early 
Start and non-Early Start) were equivalent after the weights were applied. 
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Early Start Participation 


Summer QR placement 


With the implementation of EO 1110, the CSU adopted the use of multiple measures to 
designate each student’s readiness for college-level work, known as the student’s placement 
status. The policy calls on campuses to use a variety of measures, including high school grades 
and test scores, to designate students in one of four placement categories (Box 1). 


Box 1 — California State University placement categories, based 
on multiple measures 


Category I: Has fulfilled the requirement for General Education (GE) Subarea A2 (for Written 
Communication, or WC) or B4 (for mathematics/quantitative reasoning, or QR) 


e Student has met the CSU GE Breadth Subarea A2 and/or B4 requirement via Advanced 
Placement examination, International Baccalaureate examination, or transferable 
course. 


Category II: Placement in a GE Subarea A2 or B4 course 


e Student has met examination standards and/or multiple measures—informed 
standards. 


Category Ill: Recommend placement in a supported GE Subarea A2 or B4 course 


e Based on new multiple measures, student needs additional academic support. 


e Participation in the Early Start program is recommended and may be highly advisable 
for some students, particularly STEM majors. 


Category IV: Require placement in a supported GE Subarea A2 or B4 course or the first term of 
an applicable stretch course 


e Based on new multiple measures, student needs additional academic support. 
e Participation in the Early Start program is required. 
Note: Placement categories for WC and QR courses are determined by a combination of student grades and test scores. For a 


detailed description of the various ways in which a student can be placed into the different categories, see 
https://calstate.policystat.com/policy/6656541/latest, 


Student placement designations prior to the summer term dictated whether a student was 
“recommended” or “required” to enroll in Early Start. Students who went on to complete a B4 
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requirement in the summer were then recategorized into Category | at the start of the fall 2019 
term. Therefore, whereas WestEd’s previous report looked at student placement in the fall 
term (Bracco et al., 2021), this analysis looks at placement designations at the outset of the 
summer term. The report first describes the demographic makeup of the students enrolled 
across the seven campuses placed in QR Category III or IV in summer 2019 and then further 
describes the characteristics of those students who ultimately participated in Early Start. 


A total of 24,616 students enrolled in one of the seven CSU campuses as first-time students in 
fall 2019. Of this total, 16 percent (3,844) were placed in QR Category III and 14 percent (3,523) 
were placed in Category IV at the start of the summer session (Figure 1). 


Figure 1. QR placement at the beginning of summer 2019 (admission), first-year 
entering students in fall 2019 


Category IV, 
14% Category I, 12% 


Category Ill, 
16% 


Category Il, 
58% 


Note: Based on 24,616 first-year students enrolled at one of the seven CSU campuses included in this analysis during the fall 
2019 term. For one campus, summer placement data were not available at the time of this analysis and were estimated 
based on a combination of GPA and SAT/ACT information (see appendix A). Frequency counts are presented in Appendix B, 
Table B1. 


For the purposes of Early Start, this study is most interested in students designated as needing 
additional supports for their math courses, meaning those who were placed in QR Categories III 
and IV. Box 2 shows the demographic breakdown for students in these categories for the 2019 
summer term: 62 percent are female, 76 percent are Hispanic or Latino, and more than 26 
percent had declared a STEM-related major (including Health-Related STEM). 
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Box 2 — Cohort demographics for students in QR Categories III 
and IV (as of summer 2019) 


62% Female 
Race/Ethnicity 
e 76% Hispanic/Latino 
e 7% Asian 
e 7% Black/African American 
e 5% White 
e 5% Other 


Declared Majors 


e 46% Not STEM 

e 24% STEM (excludes Health-Related STEM) 
e 14% Undeclared 

e 13% Business 


e 3% Health-Related STEM 


Note: “Asian,” “Black/African American,” “White,” and “Other” refer to Non-Hispanic students. The “Other” category includes 
American Indian, Alaska Native, Native Hawaiian, and Other Pacific Islander. Percentages in this box are based on the 7,367 
students who were placed in QR Categories III or IV. This analysis uses the IPEDS Race/Ethnicity reporting categories. The CSU 
uses a slightly different method to report race/ethnicity in which any students who are non-U.S. citizens with an “F,” “J,” or 
“other” visa or who are non-U.S. citizens with no visa or undetermined status are put into their own category as International 
Students/Non-Resident Alien Students. Additional demographic details are presented in Appendix B, Tables B2—B4. 


Participation in Early Start, by student characteristics 


Although EO 1110 states that all students placed in Category IV are “required” to enroll in Early 
Start, not all students did so, presumably for a variety of reasons. Campus interviewees noted 
that some students may have faced too much difficulty getting to campus in the summer while 
also trying to work and save money for the upcoming semester. Additionally, some students 
were designated as Category IV in both QR and WC. According to EO 1110, although students 
could choose to enroll in Early Start in both subject areas, they were only required to choose 
one. Although campus leaders shared that they would likely encourage students to enroll in QR 
if they were placed in Category IV in both subject areas, the choice was ultimately up to 
students. 
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Participation in Early Start, by placement category 


Twenty-two percent of students in QR Category Ill and 57 percent of students in QR Category IV 
enrolled in an Early Start QR course in the summer of 2019 (Figure 2). The higher percentage of 
Category IV students enrolling in Early Start may be reflective of the fact that these students 
were “required” to enroll, whereas participation was only “recommended” for students in 
Category Ill. 


Figure 2. Participation in Early Start, by placement category, for students in QR 
Categories III and IV, 2019 cohort 


n = 838 
Cat II 
n = 2,003 
Cat IV ! 


Note: Based on the 7,367 students in QR Categories III and IV at admission (i.e., at the onset of the summer 2019 term). 
Frequency counts are presented in Appendix B, Table B5. 


Participation in Early Start, by gender 


A slightly higher percentage of males (41%) in Categories III and IV participated in Early Start QR 
courses in the summer of 2019, compared to their female counterparts (37%) (Figure 3). 
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Figure 3. Participation in Early Start for students in QR Categories III and IV, by gender, 
2019 cohort 


FEMALE 37% 


Note: Based on the 7,367 students in QR Categories III and IV at admission (i.e., at the onset of the summer 2019 term). The 
figure does not include the 10 students who did not indicate female or male as their gender and were placed in an “Other” 
category. Frequency counts are presented in Appendix B, Table B6. 


Participation in Early Start, by race/ethnicity 


Earlier research (Bracco et al., 2020, 2021) indicates that Hispanic/Latino and Black/African 
American students are overrepresented in Categories Ill and IV relative to their representation 
in the larger first-year cohort in the CSU system. Whether or not these students were 
specifically encouraged to enroll in Early Start, a higher percentage of Hispanic/Latino (42%) 
and Black/African American (33%) students in Categories III and IV participated in Early Start in 
the summer of 2019 than their Asian (24%) or White (21%) counterparts (Figure 4). These 
differences were most pronounced when looking at Early Start participation for Category IV 
students only: 61 percent of Hispanic/Latino students and 46 percent of Black/African American 
students who were designated as QR Category IV participated in Early Start, whereas 39 
percent of Asian and 36 percent of White students in this category participated.’ 


7 Frequency counts for Category IV students’ Early Start participation, by ethnicity, are in Appendix B, Table B7.2. 
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Figure 4. Participation in Early Start for students in QR Categories III and IV, by 
race/ethnicity, 2019 cohort 


White 21% 


Asian 24% 


Black/African American 30% 


Other 33% 


Hispanic/Latino 42% 


Note: Based on the 7,367 students in QR Categories III and IV. “Asian,” “Black/African American,” “White,” and “Other” refer 
to Non-Hispanic students. Frequency counts are presented in Appendix B, Table B7. 


Participation in Early Start, by declared major 


Students who were in QR Categories III and IV and planned to major in a STEM field were more 
likely to enroll in Early Start than their peers who had declared other majors (Figure 5). This 
finding may be important because students in STEM fields are likely to need significantly more 
math courses than students in other majors, so the ability to get started early toward the 
completion of their B4 requirement may be especially helpful. However, this difference only 
holds when considering students in Categories III and IV taken together. When examining only 
students in Category IV, students who had not yet declared a major were the most likely to 
participate in Early Start: Approximately 62 percent of those with an undeclared major 
participated in Early Start, compared with 59 percent of non-STEM and 57 percent of STEM 
students (see Appendix B, Table B8.2). 
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Figure 5. Participation in Early Start for students in QR Categories III and IV, by 
declared major, 2019 cohort 


Business 


Health-related STEM 


Not STEM 


Undeclared 


STEM 


Note: Based on the 7,367 students in QR Categories III and IV. “STEM” excludes students in health-related STEM majors, who 
comprise their own grouping. Frequency counts are presented in Appendix B, Table B8. 


Participation in Early Start, by campus 


To understand whether trends in Early Start participation differed across the seven campuses, 
researchers examined the percentage of students in Categories III and IV who enrolled in Early 
Start on each campus. Across all seven campuses, 39 percent of students in Categories Ill and IV 
enrolled in Early Start, but this figure varied significantly by campus (Figure 6). On two 
campuses, more than 50 percent of students in these placement categories participated in Early 
Start, whereas three of the campuses had fewer than 30 percent of these students enroll in 
Early Start in the summer of 2019. (Note that these figures include students who enrolled in 
Early Start on any campus, not just the destination campus where they ended up enrolling in 
fall 2019.) The study cannot explain the differences in participation rates or suggest any bias 
that these differences imply. Further work to understand campus support for communication 
about Early Start and encouragement of attendance would be relevant. 
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Figure 6. Distribution of Early Start Participation across campuses for students in QR 
Categories III and IV, 2019 cohort 
70% 


55% 
49% 
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28% 
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Note: Based on the 7,367 students in QR Categories III and IV. Frequency counts are presented in Appendix B, Table B9. 


Percentage of Early Start participants enrolling at same campus in fall 2019 


EO 1110 allows students to enroll in Early Start at any of the 22 CSU campuses? that offer the 
program. Even with this flexible option, a vast majority of students in the sample (over 90%) 
had enrolled in Early Start on the same campus at which they enrolled during the fall semester. 
This figure varied by campus, with 61 percent to 100 percent of students who participated in 
Early Start enrolling at the same campus in the fall (Figure 7). 


® The CSU Maritime Academy does not offer Early Start courses. 
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Figure 7. Distribution of the percentage of Early Start participants taking Early Start 
classes at their campus of enrollment in fall 2019, 2019 cohort 
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Note: Based on the 2,841 students in QR Categories III and IV who participated in Early Start in summer 2019. Frequency 
counts are presented in Appendix B, Table B10. 


To understand the potential impact of the Early Start courses offered on these seven campuses, 
the remainder of the analysis looks only at those students who enrolled in the fall at the same 
campus where they had enrolled in Early Start in the summer of 2019. 


Campus course offerings 


EO 1110 provided some leeway to campuses as to the type of courses offered for Early Start as 
long as students were given the option to earn some baccalaureate credits. According to the 
May 2019 FAQ published by the Chancellor’s Office? regarding EO 1110, Early Start courses 
should meet one of the following criteria: 


e Meet the GE requirement in Subareas A2 or B4. This course can be paired with a 
workshop that carries pre-baccalaureate or baccalaureate credit. 


° https://www2.calstate.edu/csu-system/why-the-csu-matters/graduation-initiative-2025/files/academic-preparation-faq.pdf 
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e Represent the first part of a stretch sequence that, if passed, allows the student to take 
a course that meets the A2 or B4 GE requirement in the fall. This course can be paired 
with a workshop that carries pre-baccalaureate or baccalaureate credit. 


e Offer at least one baccalaureate credit and help students build skills in quantitative 
reasoning or written communication. This course can be paired with a workshop that 
carries pre-baccalaureate credit. 


Asa result of this leeway, the seven campuses in this study took different approaches to their 
Early Start offerings. Whereas most campuses in the sample offered some combination of 
classes that met the B4 requirement (the GE math requirement), other campuses offered only 
stretch classes for Early Start. On three campuses, almost all students in Early Start for QR 
enrolled in a B4 course; on two of the campuses, all students took a first semester stretch or 
prerequisite course; on the other two campuses, a mix of courses was offered, with some 
students taking stretch/prerequisite courses and others enrolling in a course that met the B4 
requirement. On some campuses, the baccalaureate courses were paired with a pre- 
baccalaureate workshop. This variation reinforces the point that program structure is 
determined at the campus level, making the consideration of systemwide outcomes 
complicated to interpret, particularly in response to EO 1110. 


Findings 


As noted in the Methodology section, this study’s findings examine outcomes on several 
indicators to compare students who participated in Early Start with peers who chose not to 
participate in Early Start but were in the same placement category and enrolled at the same 
campus. The analysis focuses on three preliminary outcomes with regard to the effects of Early 
Start participation: 


e completion of a math course of three or more units (by passing the course with a grade 
of C— or better) by the end of the fall 2019 term, 


e completion of the B4 requirement by the end of the fall 2019 term, and 
e reenrollment in the spring 2020 term. 


For math courses of three or more units and courses that satisfy the B4 requirement, the math 
completion status at the end of each semester is defined as either 


e not attempting the course, 
e attempting the course and not passing, or 


e completing the course (passing with a grade of C— or better). 
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Completion of a math course of three or more units 


As outlined in an earlier report in this series (Bracco et al., 2019), EO 1110 eliminates the use of 
pre-baccalaureate developmental math courses and shifts CSU campuses to baccalaureate-level 
courses with additional supports. In some cases, campuses opted for a two-semester sequence 
(whether through a stretch or prerequisite model) before students could meet their B4 
requirement. Given this approach, in which two semesters are needed for students to meet the 
B4 requirement, students who did not participate in Early Start would not have the option of a 
two-semester sequence but could presumably complete at least one math course. A first 
outcome of interest for this study, therefore, is whether students in Categories IIl and IV 
completed at least one three-unit math course (regardless of whether that course met the B4 
requirement). 


Completion of a math course of three or more units in summer 2019 


Most Early Start participants completed a math course of three or more units by the end of 
the summer by completing their Early Start course (Figure 8). 


e Nearly all students in Category III (98%) and most students in Category IV (89%) 
attempted a math course of three or more units by the end of the summer. 


e Asaresult, 86 percent of Early Start participants in QR Category III and 70 percent of 
Early Start participants in QR Category IV completed a math course of three or more 
units during the summer. 
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Figure 8. Completion of a math course of three or more units by the end of summer 
2019, by QR placement category, for Early Start participants 


Category Ill Category IV 
@ Attempted - Completed m@ Attempted - Not passed @ Not attempted 


Note: Based on 2,559 students in QR Categories III and IV who took an Early Start math course in summer 2019. Frequency 
counts are presented in Appendix B, Table B11. 


Completion of a math course of three or more units by end of fall 2019 


Early Start participants had a math completion status at the end of fall 2019 that was 
statistically significantly different from their non—Early Start peers in the same QR 
placement category. ’° 


Among students who did not participate in Early Start math, 74 percent of Category Ill students 
and 85 percent of Category IV students had attempted a math course of three or more units by 
the end of the fall 2019 term. By contrast, virtually all Early Start participants had attempted 
such a course, most of them during the summer (Figure 9). Students who participated in Early 
Start not only attempted a math course of three or more units at higher rates than their 
counterparts who opted out of Early Start participation, they also successfully completed those 
courses at higher rates (Figure 9). 


e For students in Category III, 53 percent of those who did not enroll in Early Start and 91 
percent of those who did enroll in Early Start completed a math course of three or 
more units by the end of fall 2019. 


e For students in Category IV, 53 percent of those who did not enroll in Early Start and 83 
percent of those who did enroll in Early Start completed a math course of three or 
more units by the end of fall 2019. 


10 X2 (2, N = 3,778) = 392.9, p< .01 for students in Category III; and X2 (2, N = 3,222) = 374.3, p< .01 for students in Category IV 


22 


WestEd @. 


Preliminary Outcomes for CSU Students in Early Start Mathematics WestEd.org 


The possibility of taking a math course of three or more units in summer 2019 gave an 
advantage to Early Start students and allowed them to reach a higher completion rate of such 
courses by the end of the fall 2019 semester. 


Although the courses offered in the summer generally have the same course name and 
description as those offered in the fall, there is no way to tell from these data whether the 
courses are similar or whether courses offered in the summer differ in a significant way. 
Students who took a math course through Early Start may have had access to additional 
academic supports, depending on the campus program, than might have been the case during 
the fall. In addition, taking a single course in the summer without the demands of an additional 
course load may explain the higher success rates in summer math. 


Figure 9. Completion of a math course of three or more units by the end of fall 2019, 
by QR placement category and by Early Start participation 


Early Start Participants Nonparticipants Early Start Participants Nonparticipants 
Category Ill Category IV 


@ Attempted - Completed Attempted - Not passed Not attempted 


Note: Based on 7,000 students in QR Categories IIl and IV across seven CSU campuses. Math completion status was 
significantly different between Early Start participants and nonparticipants for both QR placement categories 

[X2 (2, N = 3,778) = 392.9, p < .01 for Category Ill; and X2 (2, N = 3,222) = 374.3, p < .01 for Category IV]. Frequency 
counts are presented in Appendix B, Table B12. 


Completion of a B4 requirement 


By eliminating developmental courses, EO 1110 seeks to encourage students to complete the 
B4 requirement as soon as possible. Whether because they had only to complete one course 
or because they had the opportunity to complete a two-semester sequence by the end of the 
fall semester, students who participated in Early Start on the seven study campuses were 
much more likely to have completed the B4 requirement by the end of the fall term than 
their counterparts who did not enroll in Early Start. 
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B4 completion in summer 2019 for Early Start students 
A large percentage of Early Start students completed a B4 course in summer 2019 (Figure 10). 


e Fifty-seven percent of Early Start participants in Category Ill and 39 percent of 
participants in Category IV attempted a B4 course by the end of the summer. 


e Completion rates were high and, as a result, over half (52%) of Early Start participants 
in QR Category III and about one third (34%) of Early Start participants in QR Category 
IV completed a B4 course during the summer. 


Figure 10. Completion of the B4 requirement by the end of summer 2019, by QR 
placement category, for Early Start participants 


Category Ill Category IV 


m Attempted - Completed m Attempted - Not passed Not attempted 


Note: Based on 2,559 students in QR Categories Ill and IV who participated in Early Start in summer 2019. Frequency counts 
are presented in Appendix B, Table B13. 
B4 completion by end of fall 2019 


Students who participated in Early Start across the seven campuses included in this study had 
a B4 completion status at the end of fall 2019 that was statistically significantly different from 
their non-Early Start peers in the same QR placement (Figure 11).*? 


e Forty-two percent of the Category III students who did not enroll in Early Start 
attempted a B4 course by the end of the fall semester, compared to 85 percent of the 
Category Ill students who did participate in Early Start. 


1X2 (2, N = 3,778) = 559.5, p< .01 for students in Category III; and X2 (2, N = 3,222) = 573.4, p< .01 for students in Category IV 
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e Thirty-seven percent of the Category IV students who did not enroll in Early Start 
attempted a B4 course by the end of fall, compared to 78 percent of the Category IV 
students who did participate in Early Start. 


Consequently, among Category III students, 74 percent of those who had participated in Early 
Start completed the B4 requirement by the end of the fall semester, compared to 29 percent of 
those who had not participated in Early Start. Similarly, for Category IV, 60 percent of those 
who had participated in Early Start completed the B4 requirement by the end of fall 2019, 
compared to 25 percent of those who had not enrolled in Early Start. 


By offering an additional term in the summer to take and complete a B4 course before fall 
enrollment, Early Start created an opportunity for participating students to reach a higher B4 
completion rate by the end of fall 2019, compared to non-Early Start students. Further 
research is needed over a longer period to learn how completing the B4 requirement in the first 
fall of enrollment is connected to subsequent credit accumulation and retention outcomes. 


It will be particularly important to track the long-term consequences of this early completion of 
the B4 requirement for students in STEM fields. Because STEM students have several additional 
math requirements, tracking the extent to which the earlier completion of the GE requirement 
in QR helps with steady movement through subsequent math courses will be crucial to 
understanding whether this faster move through general education is beneficial for those 
students. 
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Figure 11. Completion of a the B4 requirement by the end of fall 2019, by QR 
placement category and by Early Start participation 


Early Start Participants Nonparticipants Early Start Participants Nonparticipants 


Category Ill Category IV 


@ Attempted - Completed Attempted - Not passed @ Not attempted 


Note: Based on 7,000 students in QR Categories III and IV. B4 completion status was significantly different between Early 
Start participants and nonparticipants for both QR placement categories [X2 (2, N = 3,778) = 559.5, p < .01 for Category III; 
and X2 (2, N = 3,222) = 573.4, p< .01 for Category IV]. Frequency counts are presented in Appendix B, Table B14. 


Retention 


Although students who enrolled in Early Start did have an advantage in terms of their B4 
completion rates, researchers wanted to see whether Early Start participation had any impact 
on retention rates, specifically on whether students reenrolled to continue their CSU education 
in the spring 2020 semester. There was very little difference in retention between Category III 
Early Start and non-Early Start participants (Figure 12).17 However, there is a statistically 
significant difference between reenrollment in spring 2020 for Category IV students who did 
participate in Early Start (91%) and those who did not (87%). 


22 X2 (1, N = 3,778) = 0.04, p = .84 for students in Category Ill; and X2 (1, N = 3,222) = 11.9, p <.01 for students in Category IV 
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Figure 12. Reenrollment in spring 2020, by Early Start participation and by QR 
placement category 


91% 92% 91% 
| 
Early Start Participants Nonparticipants Early Start Participants Nonparticipants 
Category Ill Category IV 


Note: Based on 7,000 students in QR Categories Ill and IV. Retention rates between Early Start participants and 
nonparticipants were not significantly different for Category III students [X2 (1, N = 3,778) = .04, p = .84] but were statistically 
different for Category IV students [X2 (1, N = 3,222) = 11.9, p < .01 for Category IV]. Frequency counts are presented in 
Appendix B, Table B15. 


This result was confirmed by the matching analysis. Compared to Early Start participants, 
similar students placed in Category IV who did not take Early Start classes had a lower retention 
in spring 2020 (91% compared to 88%), and the difference was statistically significant at the 

1 percent level (see Appendix A). This result means that Early Start had a positive impact on 
short-term retention, and this impact was unlikely to have happened by chance. *? 


WestEd’s most recent prior report on EO 1110 (Bracco et al., 2021) found that the larger 
retention drops tend to happen between the second and third semesters. A longer-term 
analysis is needed to study whether these drops continue or worsen in subsequent semesters. 


Observations and Conclusions 


This analysis of short-term outcomes of Early Start participation on seven campuses shows that, 
after a single semester, enrolling in the new Early Start courses had a positive impact on both 
students’ ability to complete a baccalaureate-level math course of three or more units and their 


13 The estimated standardized effect of Early Start participation on retention was estimated as the odds ratio for participation 
in a regression model predicting retention that included as covariates the same variables that were used in the matching 
process. The estimated odds ratio of 1.44 means that the probability of reenrolling was 1.44 times higher for Early Start 
participants compared to nonparticipants. The variables were campus of enrollment, QR placement, high school GPA, high 
school math GPA, major of admission, age, gender, and ethnicity. 
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ability to complete the B4 requirement (the General Education requirement for math) by the 
end of the fall term. For students in Category IV (those determined to have the highest need for 
additional academic support), there was also a small, statistically significant short-term impact 
on retention, as those students had a slightly greater likelihood of reenrolling in the spring than 
their counterparts who had not enrolled in Early Start. 


Although these short-term outcomes are positive, it is too soon to tell whether these benefits 
will have an impact on retention and completion in the long run. Additionally, this analysis 
cannot determine whether the success in the summer courses was due to the focused time on 
one course, the additional supports provided in the summer, and/or a difference in the courses 
themselves. What does seem clear, however, is that allowing students to take baccalaureate- 
level courses in the summer does more for their progress than simply offering a single-unit pre- 
baccalaureate course. Further research might also consider the potential short- and long-term 
effects of participating in Early Start on a different campus than the one at which a student 
enrolls in the fall. 


Gathering programwide information would be an important next step in this analysis. Having 
data from just seven campuses is not enough for determining differences between programs in 
terms of their encouragement to enroll in Early Start, student engagement, tutorial supports, 
peer learning situations, and so on. All of these details are important and have the potential to 
impact the benefits of Early Start at the campus level. Anecdotal information that researchers 
for this study have collected over the past several years about Early Start indicates significant 
variation in how the Early Start program is implemented across different CSU campuses and 
even within the seven campuses that are the focus of this analysis. In that sense, referring to 
the Early Start “program” is somewhat of a misnomer, and the results of this study may suggest 
opportunities for strengthening Early Start at some campuses and coordinating how it is 
implementated across the CSU system. 


As noted throughout this report, interpreting this study’s findings requires an extra reminder of 
not only the short duration between students’ summer 2019 Early Start experiences and the fall 
2019 course outcomes but also a recognition that selection effects played a role in two critical 
ways. First, the identification of just seven campuses (described earlier) was intentional as a 
way of testing an Early Start implementation model that was perceived to have been largely 
responsive to the policy intent of EO 1110. Many of these campuses had offered baccalaureate- 
level math courses during the summer 2018 term and so potentially had additional time to 
experiment with course design for summer 2019. These campuses may also have been better 
situated than some to offer in-person Early Start courses because of the geographic proximity 
of their students. 


Second, the relationship between the summer Early Start experience and fall 2019 outcomes 
was built analytically on a self-selected subsample of students. These students chose to 
participate in Early Start in response to the recommendation for Category III students and the 
mandate for Category IV students to attend the program. The bias resulting from this self- 
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selection cannot be quantified but is worth further investigation at the campus level. Which 
students, for what reasons, on each campus are inclined to take up the offer to participate in 
Early Start in the summer before their first semester? How are those students’ experiences 
correlated with math preparation at the end of high school? Moving forward, such selection 
biases create opportunities to reach further into the campus-specific and student-specific 
characteristics as policy and programming discussions continue around Early Start investments. 


The abrupt move to remote learning in March of 2020 and the continuation of this mode 
throughout the 2020/21 academic year may make it difficult to fully determine the long-term 
impacts of Early Start in 2019. Individual campuses, however, may be able to determine the 
extent to which the components of their Early Start offerings are sufficient for providing more 
than an initial acceleration toward completing the B4 requirement. It will be important to 
examine the extent to which campus faculty as well as those who work with student success 
efforts assess the success of Early Start efforts that are focused primarily on providing students 
the opportunity to obtain baccalaureate credit (including the opportunity to fully complete the 
B4 requirement, in some cases) before enrolling in the fall semester. Although some campuses 
may have approached Early Start as a more comprehensive program that includes supports to 
help acclimate students to the campus and to provide additional counseling and guidance 
related to student success, EO 1110 does not require campuses to include such additional 
nonacademic supports. 
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Appendix A: Methodology 


Population of Study 


The findings in this report are based on WestEd researchers’ analysis of data on course 
enrollment provided by the California State University (CSU) Chancellor’s Office for the cohort 
of first-time students entering the CSU system in the fall 2019 term. A total of 24,616 students 
entered the CSU system as first-year students in fall 2019 across the seven campuses selected 
in collaboration with the Chancellor’s Office to be the focus of this study because of their 
robust implementation of Early Start math courses. The seven campuses included in this study 
all had at least 100 students enrolled in Early Start math, and all offered baccalaureate-level 
courses. 


The analysis included in this report focuses on 7,367 students placed in Categories III and IV in 
Mathematics/Quantitative Reasoning (QR) at the beginning of summer 2019. To focus the 
analysis on the impact of taking Early Start courses, the findings were next limited to students 
who took Early Start in summer 2019 at the same campus where they then enrolled in fall 2019 
and to non-Early Start students. After further restricting to students with non-missing grade 
point average (GPA) data, the study reports on 7,000 students, including 2,559 Early Start 
participants. 


Key Definitions 


Math Early Start participation 


Math Early Start participation was defined as taking an Early Start math class at one of the 
seven sample campuses and enrolling in fall at the same campus. 


Summer placement 


Summer placement information was provided by the CSU Chancellor’s Office for all students 
enrolling at six of the seven sample campuses in fall 2019. For one campus, summer placement 
was not available at the time of this study and instead was estimated based on a combination 
of overall GPA, high school math GPA, and SAT/ACT information. In addition, for 163 Early Start 
participants, summer placement was missing. It was imputed from their fall placement value 
(63 students in Category Ill and 100 students in Category IV). 
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Race/ethnicity 


This study used the IPEDS Race/Ethnicity reporting categories. CSU uses a slightly different 
method to report race/ethnicity, in which any students who are non-U.S. citizens with an “F,” 
“J,” or “other” visa, or who are non-U.S. citizens with no visa or have undetermined status, are 
put into their own category as International Students/Non-Resident Alien Students. 


Declared major 


The declared major category reported in this analysis was derived based on the Concentration 
Grouping provided by the CSU Chancellor’s Office. An indicator, STEM/Not STEM, was included 
in the matching that regrouped Health-related STEM concentration and STEM majors versus all 
other concentrations (Business, Not STEM, and Undeclared). 


Matching Analysis 


This study employed a matching analysis using the Mahalanobis distance metric (see, for 
instance, Rosenbaum & Rubin, 1983; Imbens, 2015). 


Matching process 


The process entailed analytically matching Early Start participants with similar students who did 
not take Early Start classes in summer 2019 based on the Mahalanobis distance metric, which is 
defined as the distance Dj, between two values of the covariate vector xj and x;: 


Dy = (Xi- Xj)’ 22 (Xi-X) 


where > is the sample covariance matrix of the covariates. The matching was conducted in 
SAS® statistical software (SAS/STAT® 14.2), using the PROC PSMATCH. The following variables 
were used to compute the Mahalanobis distance metric between students: campus of 
enrollment, QR placement (summer), overall high school GPA, high school math GPA, major of 
admission (STEM/Not STEM), age, gender, and race/ethnicity (IPEDS). Students were only 
matched within the same campus. 


In the matching analysis, each Early Start student was matched to the most similar non—Early 
Start student (“nearest neighbor”) such that the matching was conducted “one-to-one.” The 
matching procedure only matched Early Start students to similar non—Early Start students. 
Thus, the resulting estimate is the average treatment effect for the treated subjects (Steiner & 
Cook, 2013). Matching was conducted “with replacement” such that each non—Early Start 
student could be a match to multiple Early Start students if that non—Early Start student was 
similar to multiple Early Start students. For analysis, each Early Start student received a weight 
of 1, and each matched non-Early Start student received a weight that was proportional to the 
number of times the student was selected as a match. The total sample sizes of each group 
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(Early Start and non-Early Start) were equivalent after the weights were applied (2,559 Early 
Start students and 2,559 non-Early Start students). 


Baseline balance testing 


Baseline balance testing is necessary to determine whether the samples of Early Start students 
and non-Early Start students included in the analysis are similar. This testing is conducted on 
the final analytic sample after the matching has been performed using weighted frequencies. 
Baseline distributions of gender, QR placement, race/ethnicity, overall high school GPA, and 
high school math GPA were compared between the two groups. Weighted standardized 
differences between treated and untreated subjects are reported in Table A1. A standardized 
difference of 0.1 or more typically denotes a notable imbalance in the baseline covariate — that 
is, a difference between the treated and untreated subjects in the matched sample (Austin, 
2009). Table A1 shows the standardized mean differences on weighted matched observations 
with respect to QR placement, gender, age, race/ethnicity, high school GPA, and high school 
math GPA; all values are largely less than 0.1, indicating negligible difference between the two 
groups of students. 


Table A1. Standardized mean difference on weighted matched observations 


Standardized Mean Difference on Weighted Matched Observations 


Placement in Category IV 0.016 
Female -0.008 
Ace 0.034 
White 0 
Black/African American 0 
Asian 0 
Overall High School Grade Point Average 0.039 
High School Math Grade Point Average 0.008 
STEM Concentration 0.010 
Campus 0 


Note: Standardized mean differences computed for 2,559 Early Start participants and 2,559 weighted matched non-Early 
Start participants. 


Outcome analysis after matching 


After each Early Start student was matched to the most similar non—Early Start student, all of 
the matched students were included in a logistic regression model that included the same 
variables as covariates that were used in the matching process. This process was done to make 
the evaluation more robust in that the matching and the regression protects against 
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misspecification in either model (Imbens & Wooldridge, 2009). Previous studies have suggested 
that matching on a set of baseline data that are strongly predictive of the outcome measure 
and then using regression methods on the matched sample can succeed in replicating 
experimental impacts in certain contexts (Cook et al., 2008). 


The regression model was of the following form: 
Pr(reenrollment; = 1) = logit +(B0 + B1 * Demographics) + B2 * EarlyStart; + €) 


where the subscript refers to student i, Demographics; represents the same vector of student 
characteristics used for matching (campus of enrollment, QR placement, overall high school 
GPA, high school math GPA, major of admission, age, gender, and race/ethnicity), EarlyStart; 
represents each student participation in the program, € represents the error term, and the 
coefficients B are estimated from the data. The function logit™*(z) = e?/(1 + e”) transforms 
continuous values to the range (0,1) and is used because the dependent variable for 
reenrollment is binary. 


In addition, because non—Early Start students could be included multiple times in the regression 
analysis, cluster-robust standard errors (Huber, 1967) were used in order to allow for 
intragroup correlation at the individual level. The use of robust standard errors when 
conducting a regression after matching with replacement is suggested by Hill and Reiter (2006, 
p. 2234). Table A2 displays the results from the logistic regression analysis that includes the 
matched Early Start and non—Early Start students. 


The estimate for the Early Start enrollment variable in Table A2 represents the impact estimate 
for the Early Start participation. Non—Early Start is the reference category and the estimate 
(0.369) for Early Start students is positive and statistically significant at the 1 percent level. In 
other words, compared to not taking Early Start courses, enrolling in Early Start was found to 
have a positive impact on student reenrollment in spring 2020, and this impact was unlikely to 
have happened by chance. 


The estimated effect size of the Early Start participation is provided by the Odds Ratio for Early 
Start program participation. An odds ratio (OR) is a measure of association between an 
exposure and an outcome. The OR represents the odds that an outcome (reenroll in spring 
2020) will occur given a particular exposure (Early Start), compared to the odds of the outcome 
occurring in the absence of that exposure (non—Early Start). OR=1.44 [1.11, 1.89]. The 
estimated odds ratio of 1.44 means that the probability of reenrolling was 1.44 higher for Early 
Start participants compared to nonparticipants. 
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Table A2. Logistic regression analysis showing the impact of Early Start participation 
on reenrollment in spring 2020, 2019 cohort 


Parameter Estimate iio} oLUKsi ms }e-] ale l=] gem =a cele 
Intercept 4.876 ** 2.1287 
Early Start 0.369 ee 0.1356 
Early Start Participation oases Sart 0.000 0 
Placement Category IV -0.001 0.1778 
Campus B 0.019 0.2352 
Campus G alata ae 0.5063 
Campus C 0.812 +e 0.2806 
Campus Campus A 0.623 aa 0.2769 
Campus E 0.782 a 0.3139 
Campus D -0.480 0.3079 
Campus F 0.000 0 
Gender Female 0.177 0.1497 
Age =0'303;, || *** 0.1033 
Asian 0.491 0.5304 
Black/African American -0.571 0.5203 
Race/Ethnicity Hispanic/Latino -O1342 0.4334 
Other/Multiple/Unknown -0.150 0.4971 
White" 0.000 0 
High School Grade Point 0.011 cael 0.0034 
Average 
Math High School -0.003 = 0.0018 
Grade Point Average 
Major of Admission STEM Concentration -0.034 0.1750 


Note: * denotes statistical significance at the 10 percent level; ** denotes statistical significance at the 5 percent level; 
*** denotes statistical significance at the 1 percent level. " indicates the reference category. Weighted number of 
observations: 5,118. Robust standard errors calculated using SAS/STAT® PROC GENMOD. 
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Appendix B: Data Tables 


Table B1. Distribution of QR placement at the beginning of summer 2019 (admission), 
first-year entering students in fall 2019 


Placement category Number of students | Percent | 


Category | 2,876 12% 
Category Il 14,373 58% 
Category Ill 3,844 16% 
Category IV S523 14% 
Total 24,616 100% 


Note: The numbers presented in Table B1 are shown in Figure 1 in the main report. The numbers in the main report 
represent 30 percent of students who were placed in Category III or IV. 


Table B2. Number and percentage of students in QR Categories Ill and IV, by gender 


Female 4,530 62% 
Male 2,827 38% 
Other 10 0% 


Total 7,367 100% 


Note: The numbers and percentages shown in Table B2 are presented in Box 2 in the main report. The numbers represent 
the total number of students in QR Categories III and IV from the 2019 cohort. Category “Other” refers to students who had 
neither male nor female as their reported gender. 
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Table B3. Number and percentage of students in QR Categories III and IV, by 


race/ethnicity 
ae 

Hispanic/Latino 5,619 76% 

Asian 548 7% 

Black/African American 482 7% 

White 367 5% 

Other/Multiple/Unknown 351 5% 

Total 7,367 100% 


Note: Asian, Black/African American, White, and Other represent Non-Hispanic students. The numbers presented in 
Table B3 are shown in Box 2 in the main report. 


Table B4. Number and percentage of students in QR Categories III and IV, by 
declared major 


Declared major Number of students | Percent 


Not STEM 3,408 46% 
STEM (excludes health-related STEM) 1,754 24% 
Undeclared 1,039 14% 
Business 947 13% 
Health-related STEM 219 3% 
Total 7,367 100% 


Note: The numbers presented in Table B4 are shown in Box 2 in the main report. 
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Table B5. Early Start participation by summer QR placement distribution for students 
in QR Categories III and IV 


Early Start versus 
non Early Start 
N % N % N % 


Placement category o o o 


Category Ill 838 22% 3,006 78% 3,844 100% 


Category IV 2,003 57% isy20) 43% 3,523 100% 


Note: The numbers and percentages from Table B5 are represented in Figure 2 in the main report. N represents number of 
students and % percentage. 


Table B6. Early Start participation by summer QR placement for students in Categories 
lll and IV, by gender 


Early Start versus Non Earl Non Earl 
Y Early Start Early Start Y Y a Ke)e-)| ae) -) | 
non Early Start Sita Start 
N % N % 


Gender N % 

Female 1,670 37% 2,860 63% 4,530 100% 
Male 1,164 41% 1,663 59% 2,827 100% 
Other 7 70% 3 30% 10 100% 


Note: The numbers presented in Table B6 are shown in Figure 3 in the main report. The table shows all students assigned to 
QR Categories Ill and IV, by gender. N represents number of students and % percentage. 
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Table B7. Early Start participation by summer QR placement for students in Categories 


Ill and IV, by race/ethnicity 


mcm [EY | exiysn 
Race/Ethnicity N % 
Hispanic/Latino 2,370 42% 
Asian i132 24% 
Black/African American 145 30% 
White 78 21% 
Other/Multiple/Unknown 116 33% 


Non Early 


Start 


3,249 


416 


333)// 


289 


235 


Non Early 
Sitlad 


% 


58% 


76% 


70% 


79% 


67% 


Total 


482 


367 


351 


Total 


% 


100% 


100% 


100% 


100% 


100% 


Note: Table B7 shows all students in QR Category III or IV as of the beginning of summer 2019. Numbers presented here can 


be found in Figure 4 in the main report. N represents number of students and % percentage. 
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Table B7.1. Early Start participation for students in summer QR Category III, by 
race/ethnicity 


(o]; Wer) «-1-40) aa || Early Start | Early Start a Ke) e-) | ae) -) | 


Race/Ethnicity N % N % N % 

Hispanic/Latino 721 25% 2,209 75% 2,930 100% 
Asian 25 9% 252 91% 277 100% 
Black/African American 34 14% 205 86% 239 100% 
White 19 9% 185 91% 204 100% 
Other/Multiple/Unknown 39 20% 155 80% 194 100% 


Total 838 22% 3,006 78% 3,844 § 100% 


Note: Table B7.1 shows all students in QR Category Ill as of the beginning of summer 2019. N represents number of students 
and % percentage. 
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Table B7.2. Early Start participation for students in summer QR Category IV, by 


race/ethnicity 


QR Category IV 


Race/Ethnicity N % 

Hispanic/Latino 1,649 61% 
Asian 107 39% 
Black/African American 111 46% 
White 59 36% 
Other/Multiple/Unknown 77 49% 


Early Start Early Start 


1,040 


164 


32. 


104 


80 


39% 


61% 


54% 


64% 


51% 


27 


243 


163 


Sy 


100% 


100% 


100% 


100% 


Note: Table B7.2 shows all students in QR Category IV as of the beginning of summer 2019. N represents number of 
students and % percentage. 
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Table B8. Early Start participation by summer QR placement for students in Categories 
lll and IV, by declared major 


Early Start versus non Early Start 


Declared major N % N % N % 

Not STEM 1,231 36% Da ALTATL 64% 3,408 100% 
STEM (excludes health-related STEM) 833 47% 921 53% 1,754 100% 
Undeclared 395 38% 644 62% 1,039 100% 
Business 307 32% 640 68% 947 100% 
Health-related STEM 75 34% 144 66% 219 100% 


Note: The numbers presented in Table B8 are shown in Figure 5 in the main report. The table shows all students assigned to 
QR Category Ill or IV, by declared major. N represents number of students and % percentage. 
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Table B8.1. Early Start participation for students in summer QR Category III, by 
declared major 


QR Category III 


Declared major N % N % N % 

Not STEM 495 23% 1,660 77% 25155 100% 
STEM (excludes health-related STEM) 72) 17% 350 83% 422 100% 
Undeclared 159 24% 501 76% 660 100% 
Business S7/ 20% ai37/ 80% 494 100% 
Health-related STEM 1S 13% 98 87% 113 100% 


Note: Table B8.1 shows all students assigned to QR Category Ill, by declared major. N represents number of students and % 
percentage. 
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Table B8.2. Early Start participation for students in summer QR Category IV, by 
declared major 


QR Category IV 


Declared major N % N % N % 

Not STEM 736 59% Syi17/ 41% 253 100% 
STEM (excludes health-related STEM) 761 57% 571 43% 1,332 100% 
Undeclared 236 62% 143 38% 379 100% 
Business 210 46% 243 54% 453 100% 
Health-related STEM 60 57% 46 43% 106 100% 


Note: Table B8.2 shows all students assigned to QR Category IV, by declared major. N represents number of students and % 
percentage. 
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Table B9. Early Start participation, by campus, for students in QR Categories III and IV 


Early Start versus 


non Early Start 


Campus 


Campus A 


Campus B 


Campus C 


Campus D 


Campus E 


Campus F 


Campus G 


N % N 


400 


838 


656 


193 


214 


378 


162 


‘o 


23% 


55% 


49% 


24% 


28% 


52% 


33% 


1,346 


678 


670 


609 


542 


345 


336 


% 


77% 


45% 


51% 


76% 


72% 


48% 


67% 


1,516 


1,326 


802 


756 


723 


498 


% 


100% 


100% 


100% 


100% 


100% 


100% 


100% 


Note: Table B9 shows all students in QR Categories III and IV as of the beginning of summer 2019, by campus. Numbers 
presented here can be found in Figure 6 in the main report. N represents number of students and % percentage. 
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Table B10. Percentage of students who took Early Start classes at the same campus as 
their campus of enrollment in fall 2019, by campus, for Early Start participants in QR 


Categories Ill and IV 
% N % N % 


Campus N 


Enrollment Fall 2019 


Campus A 314 79% 86 22% 400 100% 
Campus B 836 100% 2 0% 838 100% 
Campus C 640 98% 16 2% 656 100% 
Campus D aly) 61% 76 39% 193 100% 
Campus E 179 84% 35 16% 214 100% 
Campus F 321 85% By) 15% 378 100% 
Campus G 153 94% 9 6% 162 100% 


Note: Table B10 shows all students who participated in Early Start and were in QR Category Ill or IV as of the beginning of 
summer 2019, by campus. Numbers presented here can be found in Figure 7 in the main report. N represents number of 
students and % percentage. 
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Table B11. Completion of a math course of three or more units by the end of summer 
2019, by QR Category III or IV 


3+ units math Not Not Attempted Attempted 

Total Total 

class attempted attempted not passed not passed 

Placement — N % N % N % N % 
participation 
ts im 15 2% 97 12% 675 86% 787 100% 
Category IlI- 4.994 100% 0 0% 0 0% 2,991 100% 
Non-ES 
CategoryIV- = 492 11% 337 19% 1,243 70% 1,772 100% 
ES 
CategoryIV- = 4. 450 100% 0 0% 0 0% 1,450 100% 
non-ES 


Note: The numbers presented in Table B11 are shown in Figure 8 in the main report. The table shows all students assigned to 
QR Categories Ill and IV. N represents number of students and % percentage. 
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Table B12. Completion of a math course of three or more units by the end of fall 2019, 
by QR Category III or IV 


3+ units math Not Not Attempted Attempted 
Total Total 
class attempted attempted not passed not passed 

Placement — N % N % N % N % 
participation 
Category Ill — ES il 0% 72 9% 714 91% 787 100% 
Category Ill — 736 26% 640 21% 1,588 53% Zoi 100% 
Non-ES 
Category IV — ES 23 1% 286 16% 1,463 83% 22 100% 
CategoryIV-—non- 205 14% 471 32% 774 53% 1,450 100% 
ES 


Note: The numbers presented in Table B12 are shown in Figure 9 in the main report. The table shows all students assigned to 
QR Categories Ill and IV. N represents number of students and % percentage. 


49 


Preliminary Outcomes for CSU Students in Early Start Mathematics 


WestEd 2. 


Table B13. Completion status of a B4 requirement by the end of summer 2019, by 
QR Category Ill or IV 


B4 Not Not Attempted Attempted 
: Total Total 
requirement attempted attempted not passed not passed 


Riscement oa) WN % N % N % N % 
participation 


ts Ml- 340 43% 39 5% 408 52% 787 100% 
Category IlI- 9.991 100% 0 0% 0 0% 2,991 100% 
non-ES 

CategoryIV- == 1.077 61% 88 5% 607 34% 1,772 100% 
ES 

CategoryIV- = 4. 450 100% 0 0% 0 0% 1,450 100% 
non-ES 


Note: The numbers presented in Table B13 are shown in Figure 10 in the main report. The table shows all students assigned 
to QR Categories Ill and IV. N represents number of students and % percentage. 
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Table B14. Completion status of a B4 requirement by the end of fall 2019, by 
QR Category Ill or IV 


B4 Not Not Attempted Attempted 
: Total Total 
requirement attempted attempted not passed not passed 


Rlscement a) Win % N % N % N % 
participation 


ts im 6116 15% 86 11% 585 74% 787 100% 
Category II- —4.791 58% 388 13% 382 29% 2,991 100% 
non-ES 

CategoryIV- 336 22% 327 18% 1,059 60% iio 100% 
ES 

CategoryIV- = — 94 63% 175 12% 363 25% 1,450 100% 
non-ES 


Note: The numbers presented in Table B14 are shown in Figure 11 in the main report. The table shows all students assigned 
to QR Categories III and IV. N represents number of students and % percentage. 
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Table B15. Reenrollment in spring 2020, by QR Category III or IV 


WestEd' 2. 


Reenrolled Reenrolled Did not reenroll Did not reenroll 


Placement — N 
participation 


Category III — 720 
ES 

Category Il 2.743 
non-ES 


Category IV- 1,615 
ES 


Category IV- GL Sy) 
non-ES 


% 


91% 


92% 


91% 


87% 


67 


248 


157 


183 


% 


9% 


8% 


9% 


13% 


787 


2,991 


72 


1,450 


100% 


100% 


100% 


100% 


Note: The numbers presented in Table B15 are shown in Figure 12 in the main report. The table shows all students assigned 


to QR Categories Ill and IV. N represents number of students and % percentage. 
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